“Who writes what?” Using written comments in team-based assessment to better understand medical student performance: a mixed-methods study

BACKGROUND Observation of the performance of medical students in the clinical environment is a key part of assessment and learning. To date, few authors have examined written comments provided to students and considered what aspects of observed performance they represent. The aim of this study was to examine the quantity and quality of written comments provided to medical students by different assessors using a team-based model of assessment, and to determine the aspects of medical student performance on which different assessors provide comments. METHODS Medical students on a 7-week General Surgery & Anesthesiology clerkship received written comments on 'Areas of Excellence' and 'Areas for Improvement' from physicians, residents, nurses, patients, peers and administrators. Mixed-methods were used to analyze the quality and quantity of comments provided and to generate a conceptual framework of observed student performance. RESULTS 1,068 assessors and 127 peers provided 2,988 written comments for 127 students, a median of 188 words per student divided into 26 "Areas of Excellence" and 5 "Areas for Improvement". Physicians provided the most comments (918), followed by patients (692) and peers (586); administrators provided the fewest (91). The conceptual framework generated contained four major domains: 'Student as Physician-in-Training', 'Student as Learner', 'Student as Team Member', and 'Student as Person.' CONCLUSIONS A wide range of observed medical student performance is recorded in written comments provided by members of the surgical healthcare team. Different groups of assessors provide comments on different aspects of student performance, suggesting that comments provided from a single viewpoint may potentially under-represent or overlook some areas of student performance. We hope that the framework presented here can serve as a basis to better understand what medical students do every day, and how they are perceived by those with whom they work.